Part I: Background on LLMs
Conference Room A
- Introduction to LLMs
- Training Process of LLMs
- Root Causes of Bias in LLMs
- Training Process of LLMs
- Root Causes of Bias in LLMs
Overview
Large Language Models (LLMs) have demonstrated remarkable success across various domains over the years. However, despite their promising performance on various real world tasks, most of these algorithms lack fairness considerations, potentially leading to discriminatory outcomes against marginalized demographic groups and individuals. Many recent publications have explored ways to mitigate bias in LLMs. Nevertheless, a comprehensive understanding of the root causes of bias, their effects, and possible limitations of LLMs from the perspective of fairness is still in its early stages. To bridge this gap, this tutorial provides a systematic overview of recent advances in fair LLMs, beginning with real-world case studies, followed by an analysis of bias causes. We then explore fairness concepts specific to LLMs, summarizing bias evaluation strategies and algorithms designed to promote fairness. Finally, we analyze bias in LLM datasets and discuss current research challenges and open questions in the field.
Our tutorial is structured into five key parts:
- Background on LLMs
- Quantifying Bias in LLMs
- Mitigating Bias in LLMs
- Resources for Evaluating Bias
- Challenges and Future Directions
Check out our survey on fairness in large language models.
This tutorial is grounded in our surveys and established benchmarks, all available as open-source resources.
Speakers
Agenda
9:00 - 9:30
9:30 - 10:00
Part II: Quantifying Bias in LLMs
Conference Room A
- Demographic representation
- Stereotypical association
- Counterfactual fairness
- Performance disparities
- Stereotypical association
- Counterfactual fairness
- Performance disparities
Coffee Break
Room A5
Coffee and cakes lorem ipsum dolor sit amet, consectetur adipiscing elit.
10:30 - 11:00
Part III: Mitigating Bias in LLMs
Conference Room A
- Pre-processing
- In-training
- Intra-processing
- Post-processing
- In-training
- Intra-processing
- Post-processing
11:00 - 12:00
Part IV: Resources for Evaluating Bias
Conference Room B
- Toolkits
- Datasets
- Datasets
13:30 - 14:30
Part V: Challenges and Future Directions
Conference Room C
- Formulating Fairness Notions
- Rational Counterfactual Data Augmentation
- Balancing Performance and Fairness in LLMs
- Fulfilling Multiple Types of Fairness
- Developing More and Tailored Datasets
- Rational Counterfactual Data Augmentation
- Balancing Performance and Fairness in LLMs
- Fulfilling Multiple Types of Fairness
- Developing More and Tailored Datasets